WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
G06F 17/60 



Al 



(11) International Publication Number: WO 98/49643 

(43) International Publication Date: 5 November 1998 (05.1 1.98) 



(21) International Application Number: PCT/US98/08236 

(22) ' International Filing Date: 24 April 1998 (24.04.98) 



(30) Priority Data: 
08/845,722 



25 April 1997 (25.04.97) 



US 



(71) Applicant: POSTX CORPORATION [US/US]; 1688 Dell 

Avenue, Campbell, CA 95008 (US). 

(72) Inventors: VENKATRAMAN, R., C; 1031 Harlan Drive, San 

Jose, CA 95129 (US). CHEN, Vincent, Min-Hao; 3314 
Marchese Court, Santa Clara, CA 95051 (US). NANJA, 
Sekaran; 5824 Chambertin Drive, San Jose, CA 951 18 (US). 
GUJARATHI, Haresh; Apartment E-204, 655 South Fair 
Oaks Avenue, Sunnyvale, CA 94086 (US). 

(74) Agents: JAKOPIN, David, A. et al.; Cushman Darby & 
Cushman, IP Group of Pillsbury- Madison & Sutro, LLP, 
1100 New York Avenue N.W., Washington, DC 20005 
(US). 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA, CH. CN, CU, CZ, DE, DK, EE, ES, FI, GB, GE, 
GH, HU, ID, IL f IS, JP, KE, KG, KP, KR, KZ, LC, LK, 
LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, MX, NO, 
NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, TR, 
TT, UA, UG, UZ, VN, YU, ZW, ARIPO patent (GH. GM, 
KE, LS, MW, SD, SZ, UG, ZW), Eurasian patent (AM, AZ, 
BY, KG, KZ, MD, RU, TJ, TM). European patent (AT. BE, 
CH, CY,.DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, 
NL, PT, SE), OAPI patent (BF, BJ, CF, CG, CI, CM, GA, 
GN, ML, MR, NE, SN, TD, TG). . 



Published 

With international search report 



186 180a 

/ bubble iSxr 
If Mouse 



(54) Title: E-MAIL PROGRAM CAPABLE OF TRANSMITTING, OPENING AND PRESENTING A CONTAINER HAVING 
DIGITAL CONTENT USING EMBEDDED EXECUTABLE SOFTWARE 

(57) Abstract . . 

An e-mail message 
enhancement, apparatus is 
preferably implemented 
through a sequence of 
program instructions. On the 
sender side, creation software 
implements a data format 
that allows for differing types 
of content, such as visual 
images, audio, multimedia, 
and functional attributes 
such as links, to a sender's 
home page on the world 
wide web. Furthermore, in 
addition to such content in 
the e-mail message, there is 
included header information, 
including the e-mail address 
of the sender, and recipient 
executable embedded 
software that ensures the 
recipient will be able to view 
contents and perform the 
functional attributes that make up the e-mail message. 




200 



FOR THE PURPOSES OF INFORMATION ONLY 
Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under.the PCT. 



AL 


Albania 


ES 


Spain 


AM 


Armenia 


FI 


Finland 


AT \ 


Austria 


FR 


France 


AU 


Australia 


GA 


Gabon 


AZ 


Azerbaijan 


GB 


United Kingdom 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


BB 


Barbados 


' GH 


Ghana 


BE " 


Belgium 


GN 


Guinea 


BF 


Burkina Faso • 


GR 


Greece 


BG 


Bulgaria 


HU 


Hungary 


BJ 


Benin 


IE 


Ireland 


BR 


Brazil 


IL 


Israel 


BV 


Belarus 


IS 


Iceland 


CA 


Canada 


IT 


Italy 


CF 


Central African Republic 


JP 


Japan 


CG 


Congo 


KE 


Kenya 


CH 


Switzerland 


KG 


Kyrgyzstan 


a 


C6te d' I voire 


KP 


Democratic People's 


CM 


Cameroon 




Republic of Korea 


CN 


China 


KR 


Republic of Korea 


CU 


Cuba 


KZ, 


Kazakstan 


CZ 


Czech Republic 


LC 


Saint Lucia 


DE 


Germany 


li 


Liechtenstein 


DK 


Denmark 


LK 


Sri Lanka 


EE 


Estonia 


LR 


Liberia 



LS 


Lesotho 


SI 


Slovenia 


LT 


Lithuania 


SK 


Slovakia 


LU 


Luxembourg 


SN 


Senegal 


LV 


Latvia 


sz 


Swaziland 


MC 


Monaco 


TD 


Chad 


MD 


Republic of Moldova 


TG 


Togo 


MG 


Madagascar 


TJ 


Tajikistan 


MK 


The former Yugoslav 


TM 


Turkmenistan 




Republic of Macedonia 


TR 


Turkey 


ML 


Mali 


TT 


Trinidad and Tobago 


MN 


Mongolia 


UA 


Ukraine 


MR 


Mauritania 


UG 


Uganda 


MW 


Malawi 


US 


United States of America 


MX 


Mexico 


■ uz 


Uzbekistan 


NE 


Niger 


VN 


Viet Nam 


NL 


Netherlands 


YU 


Yugoslavia 


NO 


Norway 


zw 


Zimbabwe 


NZ 


New Zealand 






PL 


Poland 






PT 


Portugal 






RO 


Romania 






RU 


Russian Federation 






SD 


Sudan 






SE 


Sweden 






SG 


Singapore 







WO 98/49643 



PCT/US98/08236 



E-MAIL PROGRAM CAPABLE OF TRANSMITTING, 
OPENING AND PRESENTING A CONTAINER HAVING 
DIGITAL CONTENT USING EMBEDDED EXECUTABLE SOFTWARE 

Field of the Invention 

The present invention relates to an e-mail software program enhancement, and, more 
particularly, an "e-fiiailfcrogram capable of transmitting, opening and presenting a container 
having digital content (e.g. visual images, audio) using embedded --executable software. 

Background of the Invention 

Conventional e-mail systems transmit textual messages from a sender to a receiver. It is 
also customary to send attachments with messages, the attachments containing files that the 
sender desires forwarding to the receiver. 

Despite the vast usage of e-mail and e-mail systems, certain drawbacks still exist. Among 
these drawbacks include lack of privacy and verification of message receipt While encryption 
systems exist that can provide for privacy and Verification of message receipt, such encryption 
systems become tedious to properly use, and are, therefore, avoided in many circumstances due 
to the fact that they are difficult to use. Since typical e-mails are. not secure in a heterogeneous 
environment and there is no verification of message receipt, e-mail messages are typically used 
only for low priority communications. 

Furthermore, messages that are sent via one e-mail platform cannot be reliably transmitted 
to another e-mail platform and still ensure that different types of content appear as desired. For 
this and other reasons, e-mail messages typically lack interesting different types of content, such 
as visual images, audio, multimedia, and other functional attributes, such as, for,instance, links 
to a sender's home page on the world wide web. Accordingly, since only text based content 
can be easily transmitted and received in a reliable manner, most e-mail transmissions tend to 
use textual content that looks similar, with attachments of files that are similarly uninteresting. 

As a result, a more robust enhancement to conventional e-mail platforms is needed in order 
to reliably overcome the above mentioned drawbacks. 

Summary of the Invention 

It is an object of the present invention to provide an e-mail message enhancement apparatus 
that allows for privacy and verification of message receipt. 

It is another object of the present invention to provide an e-mail message enhancement 
apparatus that can be reliably transmitted from one e-mail platform to another e-mail platform. 
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It is a further object of the present invention to provide an e-mail message enhancement 
apparatus that allows for the transmission, reception and layout of interesting different types of 
content 

The above objects of the present invention, among others, are provided by the present 
5 invention through the e-mail message enhancement apparatus which is preferably implemented 
through a sequence of program instructions and achieved as attachments to e-mail message. On 
the sender side, creation software implements a data format that allows for differing types of 
digital content. Furthermore, in addition to such content in the e-mail message, there is 
included attachments, header information, including the e-mail address of the sender, and 

10 recipient executable embedded software that ensures the recipient will be able to view contents 
and perform the functional attributes that make up the e-mail message. 

In a preferred embodiment of the invention, the executable software that is transmitted as 
part of the e-mail message is written using Java programming language and contains a self- 
opening message feature, such that the container automatically opens and is initially viewed with 

15 all of the desired content appearing. Thereafter, depending upon the content originally included 
by the sender, and the action taken on the part of the recipient, different portions of the content 
and functional attributes are highlighted. . Such content and functional attributes will be 
viewable, regardless of the e-mail platform of the recipient, due to the inclusion of the 
executable software in the e-mail message itself that is transmitted. 

20 Through the use of a data structure that allows for a plurality of component objects to be 

included in a single e-mail message, as well as a plurality of sub-component objects to be 
associated with each component object, an e-mail message having robust content and functional 
attributes can be achieved. 

In a specific embodiment of the invention, the e-mail message that is transmitted has a 

25 plurality of component objects, with one of the component objects representing the front face 
of an envelope, and different other component objects including, but not limited to, recipient 
address, sender address, stamp, digital time stamp, and an envelope open attribute so that the 
e-mail message appears on the recipient's computer screen like an envelope. When the e-mail 
message is opened using the envelope open attribute, or subsequently closed, verification of 

30 message receipt is automatically forwarded back to the original sender. In a specific form of 
this embodiment, both sides of the envelope can be represented, with another component object 
representing the back side of the envelope and the envelope open attribute that allows for 
attachments or messages within the envelope to be viewed appearing on the back side of the 
envelope. ' ~ 

35 In another specific embodiment of the present invention, the e-mail message that is 

transmitted has a plurality of component objects, with one of the component objects representing 
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the front face of a postcard, and different other component objects represented on the back face 
of the postcard being the recipient address, sender address, stamp, senders letter and digital time 
stamp. ' . c 

Still furthermore, the size ancj shape of component objects can be varied as desired by the 
sender, overlap with each other and vary in number within a single e-mail message. 
Accordingly, the appearance of the e-mail message can be varied as desired by the sender. 

Brief Description of the Drawings 

The structural and functional aspects of the invention will become understood by those 
skilled in the art by reference to the following detailed description when taken in conjunction 
with the appended drawings, in which: 

FIG. 1 is a diagram illustrating an implementation of the invention in a computer network; 

FIG. 2 illustrates an e-mail message format for an enhanced e-mail message according to 
the invention; . . . 

FIG. 3 illustrates the format of an envelope data structure according to the invention; 

FIG. 4 illustrates the contents of an envelope header in an envelope data file according to 
the invention; 

FIG. 5 illustrates envelope data; 

FIG. 6 is a diagram illustrating a component objects File according to the invention; 
FIG. 7 illustrates the contents of a component object header data structure according to the 
invention; 

FIG. 8 illustrates the contents of a component object data structure according to the 
invention; 

FIG. 9 illustrates the contents of a component object property data structure according to 
the invention; . 

FIGs. 10A and 10B illustrate the process of creating and sending an enhanced e-mail 
message according to the invention; 

FIG. 11 is a diagram illustrating the process of receiving an enhanced e-mail message 
according to the invention; . ■ " ■ " 

FIG. 12 illustrates an enhanced e-mail message initial container display according to the 
invention; 

FIG. 13 is a diagram illustrating the process of handling a user action according to the 
invention; 

FIG. 14 is a diagram illustrating the process of conditional display of contents if conditions 
set by the sender according to the invention; 
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FIG. 15 is a diagram illustrating the process of sending return receipt e-mail messages 
according to the invention; 

FIGs. 16A, 16B1-16B2 and 16C1-16C4 illustrates example data structures for an envelope 
^ container having a company logo as an object component according to the invention; and 
5 FIGs. 17A-17D illustrate views and actions that result from the data structures such as set 

forth in FIGS. 16A, 16B1-16B2 and 16C1-16C4. 

Detailed Description of the Preferred Embodiment 

FIG. 1 illustrates a computer network 10 which operates upon the e-mail enhancement 

10 program according , to the present invention. A sender computer 20 communicates with a 
recipient computer 30 through a network 12. Of course, additional computers, represented as 
computers 32(1), 32(2) ...32(n) can also be connected to the network 12 and be used to send 
and receive e-mail messages. 

Sender computer 20 contains, as is known, a computing portion 22 that contains, as is 

15 known, I/O 22A, memory 22B and some type of processor 22C, which can be in the form of, 
for instance, a microprocessor. Attached to the computing portion 22 is a keyboard 24, a mouse 
26, and a display 28. Similarly, recipient computer 30 contains, as is known, a computing 
portion 32 that contains, as is known, I/O 32A, memory 32B and some type of processor 32C, 
which can be in the form of, for instance, a microprocessor. Attached to the computing portion 

20 32 is a keyboard 34, a mouse 36, and a display 38. 

Creation executable software according to the present invention contains sequences, of 
program instructions that allow for the creation of an enhanced e-mail message according to the 
present invention, as well as the transmission of the created message through the network to the 
recipient. Recipient executable software according to the present invention contains a different 

25 sequence of program instructions that allow for the receipt, and thus visual, audible and 
functional attributes to be obtained by the recipient. 

There currently exist two different embodiments of the software according to the present 
invention. In a first, version of the software, the creation and recipient executable software is 
loaded into the memory 22B of the sender computer 20. When this embodiment of the software 

30 is used, the e-mail message structure contains, as described hereinafter, a portion that attaches 
the recipient executable software to the e-mail message transmitted from the sender to the 
recipient. So long as the recipient computer 30 can operate as a Java virtual machine, the 
recipient computer 30 will receive the e-mail message containing the recipient executable 
.software, preferably written in Java, and use that recipient executable software to display the 

35 enhanced e-mail message on the display 38 of the recipient computer 30. 
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In a second, native version of the software, the creation executable software is loaded into 
the memory 22B of the sender computer 20, and the recipient executable software is loaded into 
the memory 32B of the recipient computer 32. In this embodiment, the transmitted e-mail 
message will not contain the recipient executable software, since the recipient executable 
5 software will already reside on the recipient computer 30. 

It should of course be understood that the present invention is not to be interpreted in a 
manner that limits the scope of the present invention to the specific sequence of program 
instructions. The descriptions given herein provide the teachings necessary to implement the 
present invention in different programming languages and on different operating system 
10 platforms. 

FIGs. 2-9 will now be described, which figures illustrate the data format which the Java 
executable software version implements. Those specific fields which are believed to be self 
explanatory will not be described in detail. Before describing this data format in further detail, 
. however, reference is made to FIG. 12, which illustrates an enhanced e-mail initial container 

15 display 100 made up of component objects 102, 104, 106 and 108. It should be noted that the 
number of component objects in any initial display can be varied, as can the size and shape of 
each of the component objects. Also, it should be noted that the component object is also 
referred to in certain of the drawings and potentially the specification text as a "Vixel", which 
term is a trademark of PostX Corporation. The initial display 100 provides a display that is also 

20 referred to in certain of the drawings arid potentially the specification text as "Active View", 
which term is a trademark of PostX Corporation. 

FIG. 2 illustrates the e-mail message format for the enhanced e-mail message according to 
the present invention. This data format 40 includes a native stub code 40A that provides for 
a self-opening envelope at the present time, despite the fact that Java classes are not presently 

25 defined so that self-opening can occur without the use of such a native stub. Thereafter, data 
40B indicates the number of recipient executable files, which files then follow as files 1, 2..-n, 
which are labeled as 40C-1, 40C-2..,40C-n, the number of which will be determined by the 
different number and type of images, sounds, and functional attributes that the sender inserts 
into the enhanced e-mail message that is sent. Container/envelope data file 40D follows 

30 thereafter, which file is further described with reference to FIG, 3. 

As illustrated in FIG. 3, the envelope data file 40D contains an envelope header 42, the 
contents of which are provided in detail in FIG. 4. Of note with respect to the contents of the 
FIG. 4 header is the return e-mail address of the sender, as well as the recipient name that is 
used in the return receipt e-mail function described further hereinafter, as well as an indication 

35 of whether the e-mail message is encrypted as described further hereinafter. It should be noted 
that the header" is not encrypted. 
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As also illustrated in FIG. 3, the envelope data file 40D contains envelope data 44, which 
includes the list of attachments sent by the sender, and recipients and sender's letter. Senders 
Attachments 50-1, 50-2..50-n represent attachments that are included within the enhanced e-mail 
message, as will, be described in more detail hereinafter, as will be the contents of the 
5 component objects file 54, also referred to as the Active View file, the contents of which are 
set forth in FIG. 6. 

As shown in FIG. 6, component objects file 54 includes a component object header data 
structure 56, followed by component object or Vixel data structure 58-1, 58-2....58-n, and 
component object property or Vixel property data structures 60-1, 60-2, 60-2-n,..60-n. J The 
10 contents of the component object header data structure 56 are set forth in FIG: 7, the contents 
of the component object property data structures 58 are set forth in FIG. 8, and the contents of 
sub-component object property data structures 60 are set forth in FIG. 9. 

Of note with respect to the contents of the header data structure illustrated in FIG. 7 is that 
the "platinum envelope" entry for the file information is used as a description information, no 
15 conditional processing is done with this field. The identifiers for "Container Type" correspond 
to different representations of default containers. . ■ J • 

A default "envelope" container contains component objects representing the front face of 
an envelope, and different other component objects being recipient address, sender address, 
stamp, digital time stamp, the back side of the envelope and the envelope open attribute that 
20 allows for attachments or messages within the envelope to be viewed appearing on the back side 
of the envelope so that the e-mail message appears on the recipient's computer screen like an 
envelope. 

A default "postcard" container contains component objects representing the front face of a 
postcard, and different other component objects represented on the back face of the postcard 
25 being the recipient address, sender address, stamp, and digital time stamp. 

A default "newsletter" container contains a component object representing each page of a 
newsletter, as well as various component objects on each page. 

. A default "CD" container contains a component object for each different audio track for a 
component object that has a functional attribute of playing music. 
30 A default "Rolled up Newspaper" container contains a component object representing a 

newspaper, a component object representing the index to the newspaper, and a component object 
representing each section of the newspaper, as well as^ component objects for each article if 
desired. 

A default "stack of card" container contain a component object representing each card. 
35 A default "coupon" contains a component object for each coupon that could appear in, for 

example, a newspaper. 
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The Front Color indicates the background color that will be used for the container 100 f 
whereas the Envelope Size is used by the default envelope container to represent the width and 
height dimensions. 

The Data Location refers to whether data for the component object requires looking to a 
5 data field, fetching data using a URL, or an absolute offset. The data field may be a digital data 
field that contains digital data, such as digital images from, for instance, a GIF, a video 
sequence, an audio sequence, or other representations of images. 

For each component object, there exists the data structure illustrated in FIG. 8, which data 
structure contains a Component ID that aliowsthat component object to be uniquely identified, 
10 as well as Component Attributes that identify whether the component object appears as ah 
image on the front and back of a container, or is invisible. If it appears on the front or back 
of a container, the size of the component object, its location, and the location of data are also 
specified. If it is an invisible attribute, the size and location data may not be necessary. Also 
- a hidden component object can, for instance, trigger a return receipt without the knowledge of 
15 the recipient, such that a mouse over would cause a return receipt if the hidden component 
object were configured to respond to a mouse over by providing a return receipt when a mouse- 
over occurs. 

Potentially associated with each component object identified by a data field of FIG. 8 are 
a plurality of sub-component objects. Each sub-component object associated with a component 
20 object contains data fields provided in FIG. 9. 

Each sub-component object is initiated by a User Action, which User Action is specified 
by, but not limited to the following, the position and state of the mouse pointer, which state can 
be that it is not over the component object (no action), appears over the component object 
(mouse over), there is a single mouse click when it is over the component object, or there is a 
25 mouse double click when it is over the component object. Each User Action can cause a Sub- 
Component-Type to be displayed/initiated. A plurality of sub-components can be associated 
with a single component object, and be initiated by the same or different User Action. 

Each sub-component object also has an ID, as well as an ID type that has associated with 
it a primary or a secondary sub-component object . The secondary sub-component'object will 
30 only be initiated if the primary sub-component object is unavailable, as will be described in 
more detail hereinafter. 

Having now described the preferred data format that is used to transmit the enhanced e-mail 
message according to the present invention, FIGS. 10 and 11 will be referred to in describing 
the creation and sending, and the receipt, respectively, of an enhanced e-mail message, so that 
35 a container such as the container 100 illustrated in FIG. 12 is displayed on the display 38 of a 
recipient computer 30 after having been sent from a sender computer 20. 




\ 



WO 98/49643 PCT/US98/08236 

8 

The creation and sending of an enhanced e-mail message according to the present invention, 
as well as initial receipt of an enhanced e-mail message prior to a user event, will now be 
described with respect to Figures 10 and 11. 

FIGS. 10-10B illustrate the process of creating and transmitting an enhanced e-mail message 
with a container attachment that is self opening and has the other characteristics described 
herein. In step 62, header information is first designated to define the container. Thereafter, 
component definitions are created by the sender in a step 64. There can be any number of 
component definitions desired, with it being understood that the more definitions that exist, the 
larger the size of the attachment that will ultimately be transmitted. Thereafter, sub-component 
definitions can be created for each component lobject, as described previously, is step 66. In 
step 68, the component and sub-component files are combined into an ActiveView file, which 
thereafter is combined with header information and attachments to create a container file in step 
70. The container file is encrypted in step 72. Once being encrypted, in the preferred 
embodiment, the Java class files, collectively termed the recipient executable software, and the 
native stub are attached to the encrypted container file in step 78. Mime encoding follows in 
step 80, which mime encoded file is then attached to an e-mail message in step 80 and 
transmitted. 

Upon, receipt at the . recipient computer in step 84 with reference to FIG. 11, double 
clicking the received message in step 86 if on a Java virtual machine will cause self opening 
to occurs in step 88, or, alternatively, opening the container attachment. Thereafter, the files 
are extracted from the attachment in step 90 and invocation of the application (the Java virtual 
machine and the Java class files) and extraction of the senders container data files in step 94 will 
cause the display of ttie container in step 96. Thereafter, in step 98, the computer 30 awaits the 
recipients- action, as will be further described hereinafter. 

As a result, of the display on the display 38 of the recipient computer 30, a container, such 
as the envelope container 100 illustrated in FIG. 12 is obtained. At this time, the recipient 
executable software then proceeds to a state in which it waits for the user to take an action, as 
will now be described with respect to FIG. 13. 

In step 1 10, a first event takes place. Step 112 follows and a determination is made in step 
112 as to whether the event that takes place is an event that is directed to one or more object 
components. If it is not such an event, then step 114 follows and the recipient executable 
software does nothing, and waits for a next event in step 120, Types of nothing events would 
be, for instance, if the mouse cursor 122 illustrated in FIG 12. moves to an area that is not over 
any object component. 
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If, however, step 1 12 determines that the event is directed to a component object/Vixel, then 
step 116 follows and it is determined which component objects have events, and then in step 
118 the action associated with the event intended for each component object takes place. 

For instance, if component object 102 illustrated in FIG. 12 contains an User Action of FIG. 
9 in the form of a mouse over, then the action of placing the mouse cursor 122 over the 
component object 102 will cause in step 112 a determination is made that the action involves 
a component object, and in step 116 a determination that the action pertains to component 
object 102. Thereafter, step 116 to results and the actions associated with that event will be 
executed. A specific example of such events will be provided hereinafter. 

FIG. 14 illustrates one functional attribute of the present invention, which functional 
attribute can be set forth at the initiation of receiving an enhanced e-mail message according 
to the present invention, or after the initiation of an action stemming from a component object. 
An event 124 initiates the conditional display of contents functional attribute. If this conditional 
display of contents functional attribute occurs at the initiation of the enhanced e-mail message, 
step 126 follows and the enhanced e-mail message is opened as an attachment, as has been 
previously described. Step 128 follows, in which terms and conditions which must be agreed 
upon are first made known to the recipient. Thereafter, step 130 follows and the recipients 
response is reported back to the sender, in a manner using some form of return receipt as will 
be described hereinafter. If the conditions responded to in a manner deemed appropriate by the 
sender, then step 132 follows, and the appropriate contents are then made accessible to the 
. recipient. Step 134, which follows thereafter, waits for another event to occur, as has been 
previously described with reference to FIG. 13. 

FIG. 15 illustrates another functional attribute of the present invention, referred to as. a 
return receipt. If a return receipt is requested by the sender, step 136 is initiated at an 
appropriate time. This could be when the attachment that forms the enhanced e-mail according 
to the present invention is initially transmitted to the recipient, when the recipient closes the 
enhanced e-mail that has been received, or it can be triggered by an event that relates to a sub- 
component of a component object according to the present invention. If the sender did not 
request a return receipt when initially sending the message, then step 140. follows, no action is 
taken, and then step 142 waits for another event to occur, as has been previously described with 
reference to FIG. 13. If, however, sender requested a return receipt, step 144 follows and a 
determination is made whether a specific format of response was requested. If not, step 146 
follows and a default response, such as "xyz@abc.com has opened the envelope at 10:pm 
pacific daylight time" is transmitted, this response being delivered using the sender address 
information, and other time information on the system. Step 148 follows and the computer 30 
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waits for another event to be initiated by the recipient, as has been previously described with 
reference to FIG. 13. 

If a specific format of response was requested in step 144, then step 150 follows and 
determination is made whether the sender's request requested a specific input from the recipient 
type of response. If it did not, then the senders configured response is sent in step 152 to 
addresses specified by the sender, which may be the. senders own e-mail address, the address 
of another, of the address of many others: Thereafter a step 154, like step 148 previously 
described, follows. 

If a response from recipient was requested, then the requested response is made known in 
step 156 and collected and compiled in step 158. Thereafter, the recipient's response is 
automatically transmitted in step 160 and another wait step 162 follows. 

The return receipt responses transmitted back to the sender in steps 146, 152, and 160 are 
each automatically sent using the API provided by respective e-mail clients. For SMTP, 
routines to send mail can be written using public domain information. 

HGs. 16A, 16B1-16B2 and 16C1-16C4 illustrate example data structures for an envelope 
container having a company logo as an object component according to the invention. FIGs. 
17A-17D illustrate views and actions that result -from the data structures such as set forth in 
FIGS. 16A, 16B1-16B2 and 16C1-16C4. So that the capabilities of the present invention can 
be appreciated, it should be noted that FIG. 17A is represented by the data structure of FIGS. 
16a, 16C1 and 16C2; FIG. 17B is represented by the data structure in FIG. 16A; FIG. 17C is 
represented by the data structure in FIG. 16C3; and FIG. 17D is represented by the data 
structure in FIG. 16C4. 

- Of particular note, a mouse over 180 in FIG. 17A results in a bubble 180- A, which 
corresponds to data associated with the bubble as defined by. mouse over action if FIG. 16C1 
that has a text sub-component. Similarly, FIG. 16C2 data defines that the mouse over 182 will 
cause a process launch attribute that will result in audio music being played by a program 
containing program instructions to launch the audio content, which programs can also have been 
transmitted from the sender with the originally sent enhanced e-mail message. 

"V. 

A digital time stamp 184 is also included and indicates either sender transmission time or, 
in the alternative, a certified time stamp from a dedicated time stamp server. Object 
components 188-204 are also illustrated on both sides of the envelope, and represent the ability 
of the present invention to depict, as desired by the user, many different forms, in many 
different manners, as well as attributes that then lead to the recipient to other areas. For 
instance, Figure 16C3, when a mouse click over the stamp object component 186 occurs, if the 
user is on-line at the time, will result in the user automatically being connected to the home 
page 206 illustrated in FIG. 17 A. If, however, that mouse click over the stamp object 
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component 186 occurs, but the user is off-line, then the FIG. 16C3 primary sub-component is 
not used, but the alternate sub-component identified by FIG. 15C4 is launched, which in this 
case results in a textual message being displayed. 

The above example is intended to show how robust the enhanced e-mail message 
enhancement apparatus according to the present invention actually is. By being able to be sent 
as an attachment through e-mail, the appearance of the e-mail message can be enhanced at the 
option of the sender, and the recipient will be ensured of being able to receive it in the intended 
form, since the executable recipient program instructions are preferably transmitted as part of 
the attachment. 

Furthermore, even though the executable recipient program instructions can ensure receipt 
of the message in its intended form, the architecture of the present invention also provides that 
if a more robust program or hardware element exists on the recipient computer 30, which can 
even further enhance the content that is transmitted by the sender, then that more robust 
program or hardware element exists on the recipient computer 30 can be used instead of the 
executable recipient program instructions that were transmitted by the sender. 

While the invention has been particularly shown and described with reference to the 
preferred embodiments, it will be understood by those skilled in the art that various changes in 
. form and detail may be made therein without departing from the spirit and scope of the present 
invention. - 



• 
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We claim: 

1. A method of processing an e-mail message that includes a representation of an envelope 
containing a time stamp on a computer screen of a computer, said method comprising the steps 
of: 

receiving, at said computer, said e-mail message that includes said representation of said 
envelope containing said time stamp; 

rendering on said computer screen,.using said computer, a representation of a front of said 
envelope; 

rendering on said computer screen, using said computer, a representation of said time stamp 
at a screen location that is within the front of the envelope; 

causing, upon initiation of a user action . when a cursor is positioned on said computer 
screen at a screen location corresponding to that of said time stamp, said computer to connect 
with a location associated with said time stamp,, 

2. A method according to claim 1 wherein said step of causing said computer to connect with 
said location associated with said time stamp uses a uniform resource locator to determine said 
location. 

3. A method according to claim 1 wherein said step of rendering said envelope renders said 
envelope front so that it is rectangular in shape and said time stamp is located on the upper right 
hand corner of said rectangular envelope. 

4. A method according to claim 3 wherein said e-mail message further includes a stamp, said 
stamp being displayed on said screen at the upper right hand corner of said rectangular envelope 
such that said time stamp overlaps said stamp, and said method further comprises the step of: 

causing, upon initiation of another user action when said cursor is positioned on said 
computer screen in another screen location corresponding to that of said stamp, said computer 
to connect with another location associated with said stamp. 

5. A method according to claim 4 wherein said step of causing said computer to connect with 
said another location associated with said stamp uses another uniform resource locator to 
determine said location. 

6. A method according to claim 1 wherein said step of causing is performed by execution of 
executable software that is received by said computer with said e-mail message. 



• 
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7. A method according to claim 1 wherein said e-mail message further includes textual 
content, said textual content being represented by an object on said envelope, and further 
comprising the step of causing, upon initiation of another user action when said cursor is 

, positioned on said computer screen in another screen location corresponding to that of said 
5 object, said computer to display said textual content. 

8. A method according to claim 7 wherein, upon initiation of said another user action causing 
said computer to display said textual content, said another user action also causes the steps of: 

generating an electronic response indicating said textual content has been displayed by said 
10 computer, and 

transmitting said generated electronic response. 

9. A method according to claim 1, wherein each of said envelope front and said time stamp 
have an associated component identifier which said computer uses when rendering said envelope 

15 front and said time stamp. 

10. A method according to claim 9, wherein each of said envelope front and said time stamp 
have associated component height data and component width data which said computer uses 
when rendering said envelope front and said time stamp. 

20 ^ 

11. A method according to claim 10, wherein each of said envelope front and said time stamp 
have associated vertical screen location data and horizontal screen location data which said 
computer uses when rendering said envelope front and said time stamp. 

25 12. A method according to claim 10, wherein said envelope time stamp has an associated sub- 
component identifier which said computer uses^ during said step of causing said computer to 
connect with said another location associated with said stamp. 

13. A method according to claim 1 wherein said time stamp provides both a date and a time. 
30 \ 

14. A method according to claim 13 wherein said time stamp is provided from a time stamp 
server. 
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15. A method according to claim 1 wherein said e-mail message further includes another object, 
said object being displayed on said screen arid said method further comprises the step of: 

causing, upon initiation of another user action when said cursor is positioned on said 
computer screen in another screen location corresponding to that of said object, said computer 
to produce audio tones. 

16. A method according to claim 1 wherein said e-mail message further includes ^another object, 
said object being displayed on said screen and said method further comprises the step of: 

causing, upon initiation of another user action when said cursor is positioned on said 
computer screen in another screen location corresponding to that of said object, said computer 
to play a video. 

17. A method of processing an e-mail message that includes a representation of an envelope 
containing a sender identity identification on a computer screen of a computer, said method 
comprising the steps of: 

receiving, at said computer, said e-mail message that includes said representation of said 
envelope containing said sender identity; 

rendering on said computer screen, using said computer, a representation of a front of said 
envelope; 

rendering on said computer screen, using said computer, a representation of said sender 
identity identification at a screen location that is within the front of the envelope; 

- causing, upon initiation of a user action when a cursor is positioned on said computer 
screen at a screen location corresponding to that of said sender identity identification, said 
. computer to connect with a location associated with said sender identity identification. 

18. A method according to claim 17 wherein said step of causing said computer to connect with 
said location associated with said sender identity uses a uniform resource locator to 
determine said location. 

19. A method according to claim 17 wherein said e-mail message further includes a time stamp, 
and said method further comprises the step of: 

causing, upon initiation of another user action when said cursor is positioned on said 
computer screen in another screen location corresponding to that of said time stamp, said 
computer to connect with another location associated with said time stamp. 
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20. A method according to claim 19 wherein said step of causing said computer to connect with 
said another location associated with said time stamp uses another uniform resource locator to 
determine said location. 

5 21 . A method according to claim 17 wherein said step of rendering said envelope renders said 
envelope front so that it is rectangular in shape, said sender identity identification is located on 
the upper left hand corner of said rectangular envelope and said time stamp is located on the 
upper right hand comer of said rectangular, envelope. 

10 22. A method according to claim 19 wherein said e-mail message farther includes a stamp, said 
stamp being displayed on said screen at the upper right hand corner of said rectangular, envelope 
such that said time stamp overlaps said stamp and said method further comprises the step of: 
causing, upon initiation of a further user action when said cursor is positioned on said 
computer screen in another screen location corresponding to that of said stamp, said computer 

15 to connect with another location associated with said stamp. 

23. A method according to claim 17 wherein said sender identity is represented as one of a logo 
and an address. 

20 .24. A method according to claim 17 wherein said e-mail message further includes a request for 
a configured response, said request for said configured response being represented by an object 
on said envelope, and method further comprising the steps of: 

causing, upon initiation of another user action when said cursor is positioned on -said 
computer screen in another screen location corresponding to that of said object, said computer 
25 to display said request for configured response; 

compiling of said configured response entered into said computer by said user; and 
transmitting said compiled configured response from said computer. 

25. A method of presenting, on a screen of a second computer that uses a second e-mail 
30 platform, information to a recipient that has been electronically transmitted as an e-mail message 

from a first computer using a first e-mail platform to said second computer and receiving at said 

first computer from said second computer a return receipt comprising the steps of: 

forming a data file representing said information as a plurality of objects, some of said 

objects having associated therewith digital images and certain of said objects having associated 
35 therewith a plurality of initiable sub-components, said initiable sub-components being initiated 



WO 98/49643 PCT/US98/08236 

16 

by a user action, each of said plurality of objects having an associated component identifier 
which said computer uses to rendering visual portions of said objects; 

creating executable software that can cause said information to automatically present itself 
on said screen of said second computer, generate a return receipt indicating that said information 
has been presented and cause said digital images to be rendered; 

creating an e-mail message file by combining together said executable software and said 
data file; 

electronically transmitting said e-mail message file from said first computer to said second 
computer; 

automatically presenting said information on said screen of said second computer; 

generating, at said second computer and using said executable software and said second e- 
mail platform, said return receipt; and 

receiving, after transmission of said return receipt, said return receipt at said first computer, 
thereby indicating that said information has been presented to said recipient 

26. The method according to claim 25 wherein said user action is one of a mouse-over, a 
mouse click and a mouse double click, which action occurs when a cursor is located over one 
of said certain objects and which action causes another electronic response. 

27. The method according to claim 25 wherein said user action is one of a mouse-over, a 
mouse click and a mouse double click, which. action occurs when a cursor is located over one 
of said certain objects and which action, causes audio tones to be produced. 

28. The method according to claim 25 wherein said user action is one of a mouse-over, a 
mouse click and a mouse double click, which action occurs when a cursor is located over one 
of said certain objects and which action causes a video to be played. 

•v. 

29. The method according to claim 25 wherein said user action is one of a mouse-over, a 
mouse click and a mouse double click, which action occurs when a cursor is located over one 
of said certain objects and which action causes another digital image to be presented. 
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30. The method according to claim 25 wherein said user action is one of a mouse-over, a 
mouse click and'a mouse double click, which action occurs when a cursor is located over one 
of said certain objects and which action causes said computer to connect with a location 
associated with said one of said certain objects. 

31. -A method according to .claim 25, wherein certain of said initiable sub-components will 
result in different actions, one action occurring if the second computer is on-line and another 
action occurring if the second computer is off-line. . - . 
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/* ' 
PROGRAMMER'S NAME : HARESH GUJARATHI - _ 

DATE : 06/29/1996 

PROJECT NAME : ENYE10PE.MAK 

FILENAME : ENVPHEAD.H 

HEADER DEFINITION FOR ENVELOPE FILE 

*/ 

#INCLUDE "SSDEFINE.H" 
#INGLUDE"WI«DOWS ; H" 

#IFNDEf_ENVPHEAD_H_ 
#DEFINE_ENYPHEAD_H_ 

TYPEDEF STRUCT TAGENVELOPEHEADER 

( ' . 

UNSIGNED SHORT SSIZE; 

CHAR CHMAJOR VERSION; 

CHARCHMINORVERSION; 

SHORT SDATATYPE; : 
WORD USESECURFTYATTRIBUTES: - // UINT IN 1 6 BIT ENVELOPE 
WORD USESECURITYMETHODSATTRIBUTES: //UINT IN 16 BIT ENVELOPE 
WORD UENVELOPEATTRIBUTES; //UINT IN 1 6 BIT ENVELOPE 

STR3I SZPLAYERNAME; // C STRING 

//UINT IN 1 6 BIT ENVELOPE 
// COULD BE THE URL, C-STRING 

STR3I SZSENDERNAME; // C STRING 
STR3I SZPACKAGENAME; // C STRING 

//NEW FOR HALLMARK J 
UNSIGNED LONG SRESERVEDDATALENGTH; 

UNSIGNED LONG SRESERVEDDATAOFFSET; //RESERVED DATA COMES AFTER SLIP DATA BUT BEFORE COMPONENT DATA 

FIG. 4(a) 



WORD UCOHHAND; 
STR255 SZVALUE; 



SUBSTITUTE SHEET (RULE 26) 
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UNSIGNED SHORT STEMPLATETYPE; // I: VANILLA ~ ' . 

//. 2: CUSTOM - 
UNSIGNED LONG SCOMPONENTDATALENGTH; 

UNSIGNED LONG SCOMPONENTDATAOFFSET; // COMPONENT DATA COMES AFTER RESERVED DATA 

)...,. • - 

ENVELOPEHEADER; 
ENUM ( 

SECURITYATTR_NOTSECURED = OxOOOO, 
SECURITYATTRJECURED = OxOOOl, 
SECURnYATTR_WASSECURED = 0x0002, 

); - 

ENUM ( 

ENEVELOPATTR_CLEAN = 0x0000, 
ENEVELOPATTRJENT = 0x0001 

); 

ENUM ( 
SECURITYM ETH0D_RC4 
SECURITYMETH0D_RC4_MAC 

); 

ENUM ( • 
COMMAND_LAUNCH = 0x0001, 

COMMAND JAVA = 0x002, 

COMMAND_FETCHURL = 0x0004 

): 

#ENDIF // _ENVPHEAD_H_ 



#DEFINEAPP_MINOR_VERSION0 
#DEFINEAPP_MINOR_VERSIONI 

FIG. 4(b) 



= OxOOOl, 
= 0x0002, 
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If youv'e ever asked these questions... 

. DO WE EVER HAVE SOMETHING FOR YOU 
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